Track bundle resource state sizes in telemetry (direct engine) by shreyas-goenka · Pull Request #5199 · databricks/cli

shreyas-goenka · 2026-05-06T18:00:24Z

Adds a resources_metadata field to the bundle deploy telemetry event with, per resource type, the count and the max/mean/median state size in bytes, plus the whole state file size.

eng-dev-ecosystem-bot · 2026-06-02T10:48:16Z

Commit: d0b2183

Run: 27206639157

	Env	❌FAIL	🟨KNOWN	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
🟨	aws linux		7		15	261	926	7:56
🟨	aws windows		7		15	263	924	15:15
❌	aws-ucws linux	117	1	6	15	240	840	7:48
❌	aws-ucws windows	118	1	6	15	241	838	12:39
💚	azure linux			1	17	264	924	7:00
💚	azure windows			1	17	266	922	10:50
💚	azure-ucws linux			1	17	362	836	7:38
💚	azure-ucws windows			1	17	364	834	12:07
💚	gcp linux			1	17	260	927	8:10
💚	gcp windows			1	17	262	925	10:56

140 interesting tests: 118 FAIL, 15 SKIP, 7 KNOWN

	Test Name	aws linux	aws windows	aws-ucws linux	aws-ucws windows	azure linux	azure windows	azure-ucws linux	azure-ucws windows	gcp linux	gcp windows
🟨	TestAccept	🟨K	🟨K	🟨K	🟨K	💚R	💚R	💚R	💚R	💚R	💚R
❌	TestAccept/bundle/destroy/jobs-and-pipeline	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_BUNDLE_ENGINE=direct	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_BUNDLE_ENGINE=terraform	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
🙈	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/permissions	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions	🟨K	🟨K	💚R	💚R	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct	🟨K	🟨K	💚R	💚R
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform	🟨K	🟨K	💚R	💚R
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions	🟨K	🟨K	💚R	💚R	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct	🟨K	🟨K	💚R	💚R
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform	🟨K	🟨K	💚R	💚R
🙈	TestAccept/bundle/resources/postgres_branches/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/recreate	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/replace_existing	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/update_protected	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/without_branch_id	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_endpoints/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_endpoints/recreate	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_projects/update_display_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/synced_database_tables/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/grants/select	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
❌	TestAccept/selftest/record_cloud/pipeline-crud	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestAccept/selftest/record_cloud/pipeline-crud/DATABRICKS_BUNDLE_ENGINE=direct	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestAccept/selftest/record_cloud/pipeline-crud/DATABRICKS_BUNDLE_ENGINE=terraform	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
❌	TestApiPost	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	🙈s	🙈s
❌	TestFsCat	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCat/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCatDoesNotSupportOutputModeJson	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCatOnADir	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCatOnADir/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCatOnNonExistentFile	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCatOnNonExistentFile/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCompletion	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDir	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDir/dbfs_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDir/dbfs_to_local	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDir/dbfs_to_uc-volumes	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpDir/local_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDir/uc-volumes_to_dbfs	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpDirToDirFileNotOverwritten	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDirToDirFileNotOverwritten/dbfs_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDirToDirFileNotOverwritten/dbfs_to_local	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDirToDirFileNotOverwritten/dbfs_to_uc-volumes	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpDirToDirFileNotOverwritten/local_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDirToDirFileNotOverwritten/uc-volumes_to_dbfs	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpDirToDirWithOverwriteFlag	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDirToDirWithOverwriteFlag/dbfs_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDirToDirWithOverwriteFlag/dbfs_to_local	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDirToDirWithOverwriteFlag/dbfs_to_uc-volumes	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpDirToDirWithOverwriteFlag/local_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpDirToDirWithOverwriteFlag/uc-volumes_to_dbfs	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpErrorsWhenSourceIsDirWithoutRecursiveFlag	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpErrorsWhenSourceIsDirWithoutRecursiveFlag/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDir	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDir/dbfs_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDir/dbfs_to_local	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDir/dbfs_to_uc-volumes	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToDir/local_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDir/uc-volumes_to_dbfs	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToDirFileNotOverwritten	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDirFileNotOverwritten/dbfs_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDirFileNotOverwritten/dbfs_to_local	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDirFileNotOverwritten/dbfs_to_uc-volumes	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToDirFileNotOverwritten/local_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDirFileNotOverwritten/uc-volumes_to_dbfs	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToDirForWindowsPaths	🙈s	✅p	🙈s	❌F	🙈s	✅p	🙈s	✅p	🙈s	✅p
❌	TestFsCpFileToDirWithOverwriteFlag	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDirWithOverwriteFlag/dbfs_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDirWithOverwriteFlag/dbfs_to_local	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDirWithOverwriteFlag/dbfs_to_uc-volumes	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToDirWithOverwriteFlag/local_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToDirWithOverwriteFlag/uc-volumes_to_dbfs	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToFile	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToFile/dbfs_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToFile/dbfs_to_local	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToFile/dbfs_to_uc-volumes	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToFile/local_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToFile/uc-volumes_to_dbfs	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToFileFileNotOverwritten	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToFileFileNotOverwritten/dbfs_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToFileFileNotOverwritten/dbfs_to_local	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToFileFileNotOverwritten/dbfs_to_uc-volumes	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToFileFileNotOverwritten/local_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToFileFileNotOverwritten/uc-volumes_to_dbfs	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToFileWithOverwriteFlag	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToFileWithOverwriteFlag/dbfs_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToFileWithOverwriteFlag/dbfs_to_local	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToFileWithOverwriteFlag/dbfs_to_uc-volumes	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToFileWithOverwriteFlag/local_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToFileWithOverwriteFlag/uc-volumes_to_dbfs	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToNonExistentDir	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToNonExistentDir/dbfs_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToNonExistentDir/dbfs_to_local	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToNonExistentDir/dbfs_to_uc-volumes	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpFileToNonExistentDir/local_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpFileToNonExistentDir/uc-volumes_to_dbfs	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpSourceIsDirectoryButTargetIsFile	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpSourceIsDirectoryButTargetIsFile/dbfs_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpSourceIsDirectoryButTargetIsFile/dbfs_to_local	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpSourceIsDirectoryButTargetIsFile/dbfs_to_uc-volumes	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsCpSourceIsDirectoryButTargetIsFile/local_to_dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsCpSourceIsDirectoryButTargetIsFile/uc-volumes_to_dbfs	🙈s	🙈s	❌F	❌F	🙈s	🙈s	✅p	✅p	🙈s	🙈s
❌	TestFsLs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsLs/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsLsForNonexistingDir	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsLsForNonexistingDir/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsLsOnEmptyDir	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsLsOnEmptyDir/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsLsOnFile	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsLsOnFile/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsLsWithAbsolutePaths	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsLsWithAbsolutePaths/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsMkdir	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsMkdir/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsMkdirCreatesIntermediateDirectories	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsMkdirCreatesIntermediateDirectories/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsMkdirWhenDirectoryAlreadyExists	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsMkdirWhenDirectoryAlreadyExists/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsMkdirWhenFileExistsAtPath	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsMkdirWhenFileExistsAtPath/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsRmDirRecursively	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsRmDirRecursively/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsRmEmptyDir	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsRmEmptyDir/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsRmFile	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsRmFile/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsRmForNonExistentFile	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsRmForNonExistentFile/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsRmNonEmptyDirectory	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFsRmNonEmptyDirectory/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFilerReadDir	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFilerReadDir/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFilerReadWrite	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFilerReadWrite/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFilerRecursiveDelete	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestFilerRecursiveDelete/dbfs	✅p	✅p	❌F	❌F	✅p	✅p	✅p	✅p	✅p	✅p

Top 26 slowest tests (at least 2 minutes):

duration	env	testname
6:10	azure-ucws windows	TestAccept
5:58	azure windows	TestAccept
5:52	gcp windows	TestAccept
4:54	gcp windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:50	gcp linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:38	gcp linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:29	gcp windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:42	azure-ucws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:10	aws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:08	azure-ucws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:04	aws-ucws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:03	aws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:02	aws-ucws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:01	aws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:00	aws-ucws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:57	azure-ucws linux	TestAccept
2:56	azure windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:56	gcp linux	TestAccept
2:51	azure linux	TestAccept
2:39	azure windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:39	azure linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:37	azure-ucws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:31	azure-ucws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:31	azure linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:27	aws-ucws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:25	aws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

denik · 2026-06-03T15:19:32Z

+	db := dstate.NewDatabase("", 0)
+
+	pattern := dyn.NewPattern(dyn.Key("resources"), dyn.AnyKey(), dyn.AnyKey())
+	_, err := dyn.MapByPattern(b.Config.Value(), pattern, func(p dyn.Path, v dyn.Value) (dyn.Value, error) {


Why walk the config and not the actual state? They might not match 1-1.

To capture the state for both terraform and direct deployments. Most customers are still on terraform so this givess us approximate stats for the state sizes.

I see, in that case we should not call StateFileSize, since it has nothing to do with it, we should call it ConfigFileSize.

It does try to approximate the state size - by calling PrepareState:

target := cfg if adapter, ok := adapters[resourceType]; ok { state, err := adapter.PrepareState(cfg) if err != nil { return nil, fmt.Errorf("prepare state: %w", err) } target = state } // dstate.SaveState writes resource state with MarshalIndent using these // exact prefix/indent arguments; matching them here means each resource's // byte length equals len(entry.State) on disk for direct deploys. raw, err := json.MarshalIndent(target, " ", " ") if err != nil { return nil, fmt.Errorf("marshal: %w", err) } return raw, nil

denik · 2026-06-08T12:45:49Z

+// for sub-resources like permissions / grants / secret_acls. Sub-resources are
+// tracked under the sub-resource type so they aggregate across resource
+// families. Returns "" for keys that don't match.
+func resourceTypeFromKey(key string) string {


duplicate of

bundle/config/root.go:// GetResourceTypeFromKey extracts the resource group from a resource path.
bundle/config/root.go:func GetResourceTypeFromKey(path string) string {

denik · 2026-06-08T12:46:34Z

+	resources := make([]protos.ResourceMetadata, 0, len(types))
+	for _, t := range types {
+		sizes := sizesByType[t]
+		slices.Sort(sizes)


you don't need to sort to get min/max/mean, it can be done in one pass.

We need to sort to get the median though. It's worth getting median (and later maybe P90) since mean does not give a complete picture.

Adds a `resources_metadata` field to the bundle deploy telemetry event with, per resource type, the count and the max/mean/median state size in bytes, plus the whole state file size. Only direct deploys are measured, and collection does no marshalling, file read, or JSON parsing of its own. The direct engine already serializes each resource's state during the deploy and reconstructs it via WAL replay in Finalize; ExportStateFromData now records each entry's len(state) on the ResourceState it returns. deployCore stashes that finalized state on b.Metrics, and telemetry reads the per-resource sizes straight off the in-memory map (grouping by config.GetResourceTypeFromKey). The whole-file size comes from a single os.Stat (no read/parse). Terraform stores state differently and is not collected (the field is absent there). Because the metadata is direct-only it diverges across the DATABRICKS_BUNDLE_ENGINE test matrix, so the telemetry/deploy test captures it in a per-engine out.resources_metadata.$DATABRICKS_BUNDLE_ENGINE.txt (terraform: null) and omits it from the engine-agnostic out.telemetry.txt. Per-resource sizes are deterministic for a fixed config and asserted exactly (no redaction). Only state_file_size_bytes is dropped from the golden: it is os.Stat of resources.json whose header embeds the CLI version string, which differs in length between linux/macos (0.0.0-dev+<sha>) and windows (0.0.0-dev). It is still emitted in real telemetry. The universe proto (resources_metadata, BundleResourcesMetadata, ResourceMetadata) is already merged, so this is ingested rather than dropped. Co-authored-by: Isaac

shreyas-goenka mentioned this pull request May 6, 2026

Track bundle deploy state file sizes in telemetry #5180

Closed

shreyas-goenka temporarily deployed to test-trigger-is May 6, 2026 18:00 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/state-size-telemetry branch from 4a3106b to 715f019 Compare May 6, 2026 18:07

shreyas-goenka temporarily deployed to test-trigger-is May 6, 2026 18:08 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/state-size-telemetry branch from 715f019 to 77ec7bc Compare May 6, 2026 18:32

shreyas-goenka temporarily deployed to test-trigger-is May 6, 2026 18:32 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/state-size-telemetry branch from 77ec7bc to 85dd380 Compare May 6, 2026 18:49

shreyas-goenka temporarily deployed to test-trigger-is May 6, 2026 18:49 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/state-size-telemetry branch from 85dd380 to 65bb595 Compare June 2, 2026 09:41

shreyas-goenka temporarily deployed to test-trigger-is June 2, 2026 09:41 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/state-size-telemetry branch from 65bb595 to 44e4bc0 Compare June 2, 2026 10:10

shreyas-goenka temporarily deployed to test-trigger-is June 2, 2026 10:11 — with GitHub Actions Inactive

shreyas-goenka changed the title ~~Track bundle resource counts and state file sizes in telemetry~~ stats: Record per resource state file statistics in telemetry Jun 2, 2026

shreyas-goenka requested review from andrewnester, denik and pietern June 2, 2026 10:14

shreyas-goenka marked this pull request as ready for review June 2, 2026 10:14

denik reviewed Jun 3, 2026

View reviewed changes

Comment thread bundle/phases/resources_metadata.go Outdated

shreyas-goenka requested a review from denik June 3, 2026 15:28

denik reviewed Jun 3, 2026

View reviewed changes

Comment thread bundle/phases/resources_metadata.go Outdated

shreyas-goenka force-pushed the shreyas-goenka/state-size-telemetry branch from 44e4bc0 to d16c208 Compare June 4, 2026 12:39

shreyas-goenka changed the title ~~stats: Record per resource state file statistics in telemetry~~ Track bundle resource state sizes in telemetry (direct engine) Jun 4, 2026

shreyas-goenka temporarily deployed to test-trigger-is June 4, 2026 12:40 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/state-size-telemetry branch from d16c208 to 1cab854 Compare June 5, 2026 09:55

shreyas-goenka temporarily deployed to test-trigger-is June 5, 2026 09:55 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/state-size-telemetry branch from 1cab854 to 41cc73a Compare June 8, 2026 12:38

shreyas-goenka temporarily deployed to test-trigger-is June 8, 2026 12:38 — with GitHub Actions Inactive

shreyas-goenka requested a review from denik June 8, 2026 12:40

denik reviewed Jun 8, 2026

View reviewed changes

shreyas-goenka force-pushed the shreyas-goenka/state-size-telemetry branch from 41cc73a to 4978c70 Compare June 9, 2026 11:24

shreyas-goenka temporarily deployed to test-trigger-is June 9, 2026 11:25 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/state-size-telemetry branch from 4978c70 to d0b2183 Compare June 9, 2026 12:37

shreyas-goenka temporarily deployed to test-trigger-is June 9, 2026 12:38 — with GitHub Actions Inactive

shreyas-goenka requested a review from denik June 9, 2026 13:43

denik approved these changes Jun 9, 2026

View reviewed changes

shreyas-goenka merged commit 7e85efd into main Jun 9, 2026
31 of 32 checks passed

shreyas-goenka deleted the shreyas-goenka/state-size-telemetry branch June 9, 2026 14:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Track bundle resource state sizes in telemetry (direct engine)#5199

Track bundle resource state sizes in telemetry (direct engine)#5199
shreyas-goenka merged 1 commit into
mainfrom
shreyas-goenka/state-size-telemetry

shreyas-goenka commented May 6, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Jun 2, 2026 •

edited

Loading

Uh oh!

denik Jun 3, 2026

Uh oh!

shreyas-goenka Jun 3, 2026

Uh oh!

denik Jun 3, 2026

Uh oh!

shreyas-goenka Jun 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

denik Jun 8, 2026

Uh oh!

denik Jun 8, 2026

Uh oh!

shreyas-goenka Jun 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

shreyas-goenka commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eng-dev-ecosystem-bot commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

denik Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

shreyas-goenka Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

denik Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

shreyas-goenka Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

denik Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

denik Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

shreyas-goenka Jun 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shreyas-goenka commented May 6, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Jun 2, 2026 •

edited

Loading